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PREFACE 


The Agriculture and Resources Inventory Surveys Through Aerospace 
Remote Sensing program, AgRISTARS, is a six -year program of research, 
development, evaluation, and application of aerospace remote sensing for 
agricultural resources, which began in Fiscal Year 1980. This program 
is a cooperative effort of the National Aeronautics and Space Admini- 
stration, the U.S. Agency for International Development, and the U.S. 
Departments of Agriculture, Commerce, and the Interior. AgRISTARS con- 
sists of eight individual projects. 

The work reported herein was sponsored by the Supporting Research 
(SR) Project under the auspices of the National Aeronautics and Space 
Administration, NASA. Dr. Jon D. Erickson, NASA Johnson Space Center, 
succeeded by Robert B. MacDonald, was the NASA Manager of the SR Project 
and Thomas Pendleton was the Technical Coordinator for the reported 
effort . 

The Environmental Research Institute of Michigan and the Space 
Sciences Laboratory of the University of California at Berkeley comprise 
a consortium having responsibility for development of corn/soybeans area 
estimation procedures applicable to South America within both the Sup- 
porting Research and Foreign Commodity Production Forecasting Projects 
of AgRISTARS., 

This reported research, directed at the labeling of small grains in 
multi-date Landsat data, was performed within the Environmental Research 
Institute of Michigan’s Infrared and Optics Division, headed by Richard 
R. Legault, a Vice-President of ERIM, under the technical direction of 
Robert Horvath, Program Manager, and Dr. William A. Malila, Task 
Leader . 
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1.0 

INTRODUCTION 


Crop acreage estimates made using Landsat (or indeed any remotely 
sensed data) invariably require, at some point, association of a crop 
label or labels with some sampling entity (e.g., pixel, field, cluster, 
etc.). The accuracy with which this association is made clearly has a 
substantial impact on the accuracy of the acreage estimates produced. 

In the Large Area Crop Inventory Experiment (LACIE) , the labeling step, 
which was carried out through manual analysis of imagery and associated 
information, was found to be both time-consuming and a source of con- 
siderable error. 

To the degree that sensor limitations or imperfect spectral sep- 
arability of crops are the causes of labeling errors, new techniques 
can offer little hope of substantial improvements in accuracy. How- 
ever, that portion of the labeling error associated with the labeling 
techniques themselves, or with human limitations, might be reduced 
through new approaches to labeling. An obvious candidate for Improving 
both the objectivity and the timeliness of labeling decisions is auto- 
mation of much of the labeling process. 

The technique described herein is a response to the need for a 
faster, more accurate, and more objective labeling procedure. Human 
analysts are utilized only to set up the system and provide contextual 
information which can be used to adjust the labeling procedure to local 
conditions; the labeling decisions themselves are left to the machine. 

In addition, this procedure is intended to provide a demonstration 
of some of the applications of what we have called profile technology, 
that is, the use of features derived directly or indirectly from char- 
acterizations of the continuous patterns of temporal-spectral crop de- 
velopment [1]. 
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We do not, however, put this technique forward as the final and 
best use of profile technology, but rather as a first generation tech- 
nique, a demonstration of concepts, that can be used to more fully 
understand profiles and their uses, and thereby to develop improved 
labeling techniques . 

Similarly, while fully automatic labeling procedures may be desir- 
able in terms of efficiency, the complexity of some of the decisions 
which must be made, particularly those made on a more general level, 
probably precludes replacing human analysis entirely at this time. 

Again, the technique presented here is intended to provide some start- 
ing point for the development of later procedures which can more fully 
utilize the particular contributions both the human and the machine can 
bring to the labeling process. 

Finally, the procedure as described and tested is focussed on the 
identification of spring small grains (principally spring wheat, 
barley, and oats). The underlying concepts, however, are more generally 
applicable and could be used in techniques to label other agricultural 
crops. 
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2.0 

DESCRIPTION OF PROCEDURE 


2.1 GENERAL CONCEPTS 

Basic to most labeling techniques which operate on Landsat data are 
the presuppositions that at least some crop groups or cover types ex- 
hibit distinct and characteristic temporal patterns of spectral develop- 
ment, and that at least some of these pattern differences are detectable 
at the resolution of the Landsat multispectral scanners. Manual tech- 
niques have depended on human analysts to detect these patterns and 
pattern differences from the available Landsat acquisitions [2]. How- 
ever, at least two factors hinder the ability of humans to accurately 
carry out this process. First, the Landsat observations are fairly 
widely spaced discrete samples from what are for the most part continuous 
spectral development patterns. As a result, much of the necessary 
information must be inferred from the available data. Second, samples of 
a particular crop may vary considerably, over a small region, in terms 
of stage of development and, therefore, spectral appearance on any given 
day. The combination of sparse observations and variation in development 
stage can result in samples of a single crop type showing little apparent 
spectral similarity. 

By characterizing the continuous patterns of which the Landsat ob- 
servations are samples, one can address these problems, particularly' 
though not exclusively in automated procedures. Techniques of this 
general type have achieved promising results [3]. The labeling techni- 
que presented here is another of this class of approaches. Specifically, 
it uses the characterizations of spectral development, termed profiles, 
both to adjust for planting date differences within a crop and to assign 
crop labels to unknown samples. 
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The central element in the procedure is a group of profile sets re- 
presenting spectral development of a number of crops in the domain des- 
cribed by Tasseled-Cap Greenness and Brightness (physically interpretable 
linear combinations of Landsat MSS band values - see Reference 4) . These 
profile sets were developed using spectral data from fields of known crop 
type, sampled from the U.S. Northern Great Plains over three growing 
seasons. The actual profile sets are shown in Figures 1 and 2. 

The profile sets serve as reference standards to which each unknown 
sampling entity is compared. For each profile set, a series of com- 
parisons is carried out. First, a temporal shift is determined which 
maximizes the cross-correlation of the data points to the Greenness 
profile [5]. This provides an estimate of the date of spectral emergence, 
and indirectly of the start of the growing season of the target field. 

The temporal shift estimate also provides a means of normalizing the 
planting dates of fields of a single crop type, and thereby minimizes 
one major source of spectral confusion. 

After estimating and applying the temporal shift, a multiplicative 
scale factor is computed, again using the Greenness profile. This scale 
factor is applied to normalize the magnitude of the Greenness develop- 
ment profile which is strongly influenced, within a single crop type, 
by the percentage of ground covered by green vegetation (which is itself 
influenced by such factors as planting density, fertilization, and 
moisture availability) . 

With both adjustments made, a goodness-of-f it of the data to the 
Greenness profile is computed, and similarly, using the Greenness pro- 
file temporal shift, a fit or correlation of the Brightness data to the 
Brightness profile is computed. 

The shift. Greenness fit, and Brightness correlation are used to 
compute a probability associated with the crop represented by the pro- 
file set and the sampling entity, and this combined probability serves 
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FIGURE 1. PROFILE SETS USED IN LABELER 
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as the basis for labeling decisions. More detailed description of the 
steps of the procedure is provided in Section 2.3. In a different appli- 
cation of this procedure, one might use different or additional features 
to compute the requisite probabilities. 

2.2 RECOMMENDED ENVIRONMENT 

The concept of comparing unknown samples to static representations 
of crop spectral development necessarily requires that all sources of 
spectral variation other than crop type be eliminated or minimized. As 
a result, this procedure was developed and tested using data normalized 
with respect to haze, sun angle, and sensor calibration, and screened 
for clouds, cloud shadows, etc., using algorithms developed at ERIM [4]. 

By the same reasoning, pixels that are a mixture of more than one 
crop type will not be as readily labelable as those which are pure with 
respect to crop type. In an effort to remove mixture pixels from con- 
sideration, and also to smooth some of the extraneous variability among 
pixels in a single field, an ERIM-developed spectral-spatial clustering 
algorithm, SUPERB, was used to provide sampling entities (quasi-fields 
or blobs) for labeling [6]. 

It should be expected that the performance of this or any similar 
technique would degrade if such normalization and clustering steps were 
omitted. 

2.3 SPECIFIC STEPS 

Appendix A presents a flow diagram of the major elements in the 
labeling procedure. 

2.3.1 ANALYST FUNCTIONS 

The focus of the labeler development reported here was on the 
machine aspects of the procedure. As a result, the functions assigned 
to a human analyst are only defined in general terms. 
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Acquisition Selection 

A human must decide which acquisitions to process. This selection is 
based on considerations such as apparent growing season, presence of 
clouds or haze, view angle similarity, spacing, etc. No specific process 
has been laid out for this procedure. 

Crop Calendar Adjustment 

Beginning with either historical or model-derived crop calendars, the 
human must develop estimates of the time of spectral emergence of the 
crops represented by profile sets. This is a combination of image inter- 
pretation, computer data analysis, and use of historic or expected crop 
calendar relationships between crops. While no technique has been defined 
for the specific application described in this report, a similar procedure 
has been developed for use in U.S. 1981 corn/soybeans pilot experiment 
[7]. Eventually one might utilize outputs from meteorologically based 
crop calendar models. 

The expected output of this step is estimated mean days of spectral 
emergence for all crops or crop groups represented by profile sets. 

2.3.2 PRELIMINARY PROCESSING STEPS 

Data Transformation 

All spectral data are transformed using the Tasseled-Cap transfor- 
mation [4]. Only Brightness and Greenness values are retained. 

Acquisition Check 

Both mathematics and common sense require that some minimum number 
of vegetated acquisitions be available for a sampling entity before it 
can be labeled using this technique. That minimum number is set at 
three. The determination of whether or not a particular acquisition is 


8 



vegetated is based on a simple thresholding in Greenness. "Bare soil" 
data typically range between 25 and 35 counts of Greenness (with 32- 
count offset applied) ; the lower end of the range was used as the 
threshold value. If a sampling entity has less than three acquisitions 
with Greenness greater than 25, it is labeled "unknown". 

Maximum Greenness Check 

Those targets which have at least three vegetated acquisitions are 
also required to have one acquisition with Greenness greater than 35. 

This requirement accomplishes two purposes. First, it screens out 
fields with abnormally low green vegetation development. Such fields 
are unlikely to exhibit the same spectral characteristics as healthy 
fields of the same crop. Second, it eliminates fields which, although 
they have passed the acquisition number requirement, lack acquisitions 
in the time period during which the crop is well-developed. Again, 
targets failing to meet the requirement are labeled "unknown". 

2.3.3 CROP CALENDAR EVALUATION 

(Steps described in Sections 2.3.3 through 2.3.5 are carried out 
for each of the profile sets used in the procedure.) 

Shift Estimation 

The day of maximum green development is estimated through a tech- 
nique called crop calendar shift estimation [5]. The Greenness values 
of the target are compared to the Greenness profile being considered, 
and a temporal shift is selected which maximizes a cross-correlation-like 
factor. The shift algorithm also makes additional checks on acquisition 
number and spacing. After a temporal shift based on an initial rough 
estimate of the day of peak Greenness, it checks to see that there are 
at least three acquisitions spaced more than ten days apart. 
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This constraint is the result of observations that pairs of data 
obtained from consecutive passes of different satellites do not carry 
the weight of more widely-spaced acquisitions and, in fact, tend to 
behave in the procedure as one acquisition. If a target passes this 
test and a shift is estimated, the test is applied again using the 
final shift estimate. If there are too few sufficiently-spaced ac- 
quisitions available, an "unknown" label is assigned. 

It should be noted that for this and all subsequent steps. Green- 
ness values are reduced by 25 counts. This brings the "bare soil" 
Greenness to near zero, and both downweights the importance of fitting 
at low signal values and allows for application of the multiplicative 
scaling previously mentioned. 

Probability Computation v 

The shift estimate is compared to an expected shift for the crop, 
which is derived from the expected mean day of spectral emergence, as 
provided by the analyst. The difference between these two values is 
used to assign a probability, based on an empirically-defined probabi- 
lity distribution. The distribution is a modified normal with mean 
zero and standard deviation 14 days, but with an equal probability of 
•99 assigned to all values within one standard deviation (+ or - 14 
days) to reflect the range of planting dates which are typically ob- 
served for a single crop in a sample segment. 

2.3.4 COMPARISON TO GREENNESS PROFILE 

Computation of Scale Factor 

As previously described, a multiplicative scale factor is computed 
to maximize the fit of the shifted data values to the Greenness profile. 
The scale factor is computed by 
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Scale = 


IF. 2 


1 X 


( 1 ) 


where 

F^ = profile value at time 

G. = data value at time T. 
i X 

Since planting and harvesting methods and field conditions can cause 
more variations in Greenness values at the tails of the profile, a better 
scaling can be computed if those values are excluded. Therefore the 
scale factor is based only on those acquisitions which fall at least 20 
days from either end of the profile. If one or no acquisitions meet 
this criterion, an "unknown" label is assigned. 


Greenness Profile Fit 

A chi-squared fit is computed for the scaled data and the profile. 


Fit 



( 2 ) 


where 


F^ = profile value at time 

G. = data value at time T. 
i X 

s = scale factor 

= expected variance about profile at time 


The variances used were determined using data of known crop type 
from the 26 segments comprising the developmental data set. 
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A probability is then determined for the computed fit, with degrees 
of freedom equal to one less than the number of acquisitions used. 


2.3.5 COMPARISON TO BRIGHTNESS PROFILE 

While the characteristics of Greenness profiles and crop Greenness 
development allow for relatively simple scaling and fitting, the same 
tasks using Brightness profiles are not so straightforward. Since 
Brightness is (as was intended) strongly influenced by soil character- 
istics, the early and late season portions of Brightness profiles, where 
* soil is the dominant scene component, vary considerably. In addition, 
at least some of the Brightness profiles exhibit more complex shapes 
than the Greenness profiles. Finally, variations in the amount of 
vegetative cover, which result in a simple reduction in amplitude of 
the Greenness profile, have widely varying effects on Brightness pro- 
files, since they primarily affect the amount of soil viewed. 

Since adjustment for within-crop variations in Brightness spectral 
development cannot be readily accomplished, the assumptions necessary 
for use of a chi-squared test, particularly that of a normal distribution 
of data about the mean on any given day, are not justified, and use of 
the chi-squared test is inappropriate. 

For this labeling technique, the simplifying assumption is made 
(based on empirical evidence) that the characteristic shape of the 
Brightness profiles for the various crops will be detectable most of the 
time even with the described variations in actual Brightness values. In 
order to compare overall profile shapes rather than actual values, a 
cross-correlation calculation is made as follows: 


R = 




V(lg^2)*(Ef .2) 


(3) 
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where 


= (profile value at time t^) - profile mean 
= (data value at time t^) - data mean 


and 

Profile mean 


n 

E (profile value at t.) 

i=l ^ 

n 


where 

n = number of data values used 


The probability associated with the calculated cross-correlation is 
determined using empirically derived cumulative distributions of cross- 
correlation of known grain data (from the developmental data set) to the 
grain profile. 

2.3.6 CALCULATION OF COMBINED PROBABILITY 

The three probabilities associated with each profile set are combined 
into one probability using Fisher's omnibus procedure [8]*. This test, 
which assumes independence of the individual statistics, has the advantage 
of retaining the same level of significance in the combined statistic as 
that of the individual statistics. The combined test statistic is of the 
form 

3 

T = -2 I w.lnP. (^) 

1 1 


where 

w^ = weight assigned to the i^^ test 

P^ = probability associated with the i^^ test 


Suggested by Dr. 


Jack Tubbs, University of Arkansas. 
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Using a chi-squared test, the combined probability of T is deter- 
mined, with degrees of freedom 

3 

DF = 2 I w. (5) 

i=l ^ 

The individual statistics are weighted by their importance as 
discriminants. There are no inherent constraints on the values of the 
weights and they need not sum to one. 

If the combined probability exceeds a threshold value, the crop 
represented by the profile set used is considered probable enough to be 
retained as a candidate. Otherwise, the crop is rejected. 

2.3.7 LABEL ASSIGNMENT 

When all profile sets have been evaluated, a labeling decision can 
be made. This may be either a single label based on the most probable 
profile set, or a set of probabilities associated with the ensemble of 
profile sets whose probabilities exceeded the defined acceptance thres- 
hold. The selection of one or the other of these alternatives is a 
function of the proportion estimation procedure which utilizes the 
labels . 
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3.0 

DESCRIPTION OF TEST 


A test of the labeling technique was carried out on an independent 
data set in order to evaluate and understand its performance. 

3.1 DATA SET 

A total of 38 5x6-mile sample segments were used, spanning the same 
three years as the developmental data set. The segment locations are 
shown in Figure 3. 

The SUPERB clustering algorithm was applied in its supervised mode, 
such that only pixels of the same crop type could be placed in the same 
blob. This was done to isolate the labeling performance from the effect 
of mixed blobs. However, a test of the performance on such mixed blobs 
would be a useful exercise in the future. 

3.2 ANALYSES TO BE CARRIED OUT 

3.2.1 LABELING ACCURACY 

The accuracy with which the technique identified grain and non- 
grains was determined using a number of different procedure configurations 
(see Section 3.3). For the most part, determination of accuracy was 
based on the most probable crop rather than on the set of probabilities. 
Non-grain accuracy was based on failure to choose the grain profile, 
rather than selection of the correct non-grain crop profile. 

In addition to an overall accuracy evaluation, the significance of 
effects related to growing year, agrophysical stratum unit, and number 
of available acquisitions was assessed. Year and agrophysical unit 
should affect crop spectral characteristics, relative spectral differences 
between crops, and relative crop calendar differences, all of which 
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* = 1976 
= 1977 
O = 1978 


FIGURE 3. LOCATIONS OF TEST AND EVALUATION SEGMENTS 



should affect labeling accuracies. The number of acquisitions available 
should be directly related to accuracy, since more acquisitions should 
provide a better characterization of the temporal-spectral development 
of the target field. 

3.2.2 LABELING ERROR CHARACTERIZATION 

A related area of analysis involved determining the nature of the 
errors made. Of interest were the relative probabilities of grains being 
called one of the non-grains, and vice versa. 

3.2.3 QUALITATIVE COMPONENT EVALUATION 

In addition to the more quantitative evaluations described above, a 
qualitative evaluation was included in the test to answer less easily 
defined questions related to the underlying concepts and techniques. 

These were primarily focused on the use of profiles as features, use of 
profiles as the basis for labeling procedures, and use of crop calendar 
shift . 

3.3 PROCEDURE CONFIGURATIONS 

Some elements of the labeling techniques were either modified in 
order to isolate effects of errors in various parts of the procedure or 
applied with several different parameter sets to determine optimum con- 
figurations. 

3.3.1 CROP CALENDAR ADJUSTMENT 

While the impact of errors in estimating mean days of spectral 
emergence is of great importance, it is first important to understand 
the behavior of the technique given good estimates of spectral emergence 
days. Thus ground truth information was used rather than analyst inter- 
pretation in this test. 
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Expected mean days of spectral emergence were determined for each 
segment by applying the crop calendar shift technique to all data of 
each crop type. The results of this process were histogrammed and, 
where enough data were available for a particular crop in a particular 
segment, an estimate of the central tendency of the distribution was made. 

Where too few data were available, historical relationships, results from 
similar segments, or average results for several segments were used to 
fill in the expected spectral emergence days. 

3.3.2 PROFILE SETS 

As shown in Figure 1, the entire set of profiles developed for the 
labeling procedure represents six crops or crop groups. Similarities 
among certain of the crops, however, suggest that they could be more 
frequent sources of error than others. Specifically, grasses and flax 
have spectral characteristics and development patterns that are most 
similar to the spring small grains, and are thus most likely to be con- 
fused with grain. As a result, the procedure was applied with all six 
profiles (listed in Table 1), all profiles except grass, all except 
flax, and all except grass and flax. 

Even crop profiles that are substantially different from the grain 
profile set will tend to draw some blobs away from the grain profile. 

It is therefore likely that elimination of any profile set would result 
in some increase in grain labeling accuracy. It might be possible, based 
on historical information, to omit profile sets representing crops un- 
likely to occur in the segment of interest. The results of such a pro- 
cedure were simulated by re-running the labeling procedure with the four 
profile combinations described above, but also omitting any crop (except 
grain and grass) which had five or fewer blobs in the segment (using 
ground truth information) . Grain and grass were never excluded in this 
mode since grain is the crop of interest, and grass could never be 
ruled out as a possibility. 
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TABLE 1. CROPS REPRESENTED BY PROFILE SETS 


Spring Small Grains 

Grasses 

Sunflowers 

Corn 

Soybeans 

Flax 


TABLE 2. TEST STATISTIC WEIGHTINGS EVALUATED 

(first value applies to Greenness Fit, second 
to Brightness Correlation, third to Crop 
Calendar Shift) 

1 - 0-0 1 - 1-0 1 - 1-1 

0 - 1-0 1 - 0-1 1 - 1-2 

0-0-1 0-1-1 1-2-3 
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3.3.3 TEST STATISTIC WEIGHTINGS 


As previously described, the three pieces of information used in 
making labeling decisions (crop calendar shift. Greenness Fit, and 
Brightness correlation) can be weighted to reflect confidence or impor- 
tance. Nine weightings, placing different emphasizes on the three 
factors, were used (see list in Table 2). Each factor was used 
alone, all pairs were used, and a selected group of weightings using 
all three factors was used. This last group was selected based on re- 
sults obtained in the developmental data set, which suggested that the 
shift estimate was of greatest importance, followed by Brightness and 
then Greenness. 

3.3.4 PROBABILITY THRESHOLDS 

The minimum combined probability used to select candidate profiles 
as described earlier, was set at a number of different levels: 0.0, 

.25, .33, .50, .667, and .75. One would expect higher thresholds to 
reduce errors of commission relative to the grain profile, but increase 
errors of omission. The range of thresholds was included to evaluate 
the best mix between these two error rates. 
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4.0 

TEST RESULTS AND EVALUATION 

4.1 ABILITY TO ASSIGN LABELS 

As previously described, there is a required minimum number of 
acquisitions for any particular sampling entity to be labeled. If a 
blob fails this criterion for any of the profile sets, it can only be 
labeled based on partial information. Since the grain profile spans the 
shortest amount of time of all the profiles used, it is most likely to 
be rejected from consideration due to the acquisition availability con- 
straint. If labels were assigned in cases where blobs could not 
be compared to every profile, many grains with poor acquisitions would 
be labeled non-grains. Instead, such blobs were called "unlabelable". 

Since the developmental data set tended to have more acquisitions 
available per segment, determination of the percentage of "labelable" 
blobs was based on the combined development and test data set (64 seg-^ 
ments) . For this data set, 57% of the blobs could be labeled. However, 
considered individually, most segments were either "labelable" or not. 
Sixteen of the segments had less than 20% of their blobs labeled, while 
31 of the 64 had more than 80% of their blobs labeled. 

4 . 2 LABELING ACCURACY 

Table 3 shows the grain/non-grain accuracies for the various con- 
figurations used. Grain labeling accuracies reached 86%, but errors of 
commission were also high with the same configuration. In general, 
errors of commission and omission were inversely related. 

Overall accuracies (grain and non-grain) reached 74% (Table 4) , while 
the best mix of accuracies occurred at about 69% correct for grain and 72% 
correct for non-grain. Similar results were observed in Phase 3 and Trans- 
ition Year 1978 in the LACIE [9,10] with analyst-intensive procedures. 
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TABLE 3. GRAIN/NON-GRAIN ACCURACIES 






Threshold 



U s ing A1 1 
Profiles 


0.0 



0.5 


0. 

667 

Weighting 

Grain 


Other 

Grain 


Other 

Grain 

Other 

1-0-0 

.22 


.87 

.15 


.94 

.12 

.95 

0-1-0 

.43 


.72 

.41 


.74 

.38 

.77 

0-0-1 

.35 


.75 

.35 


.76 

.35 

.76 

1-1-0 

.36 


.81 

.30 


.88 

.26 

.91 

1-0-1 

.37 


.83 

.32 


.90 

.29 

.91 

O-I-l 

.48 


.72 

.47 


.73 

.46 

.74 

1-1-1 

.47 


.79 

.41 


.85 

.39 

.87 

1-1-2 

.47 


.78 

.44 


.83 

.42 

.84 

1-2-3 

.48 


.76 

.47 


.79 

.46 

.80 

Excluding 
Grass and 




Threshold 



t ITX 

Profiles 


0.0 



0.5 


0. 

667 

Weighting 

Grain 


Other 

Grain 


Other 

Grain 

Other 

1-0-0 

.53 


.66 

.36 


.85 

.30 

.89 

0-1-0 

.59 


.54 

.52 


.65 

.45 

.72 

0-0-1 

.83 


.33 

.79 


.48 

.79 

.48 

1-1-0 

.64 


.62 

.50 


.80 

.43 

.86 

1-0-1 

.84 


.40 

, .65 


.73 

.58 

.80 

0-1-1 

.86 


.33 

.79 


.52 

.73 

.59 

1-1-1 

.85 


.41 

.68 


.73 

.62 

.79 

1-1-2 

.86 


.35 

.74 


.66 

.69 

.72 

1-2-3 

.86 


.34 

.77 


.60 

.74 

.66 


Note: "Grain" includes all blobs with Ground Truth of Spring Wheat, 

Durum Wheat, Barley, and Oats . 

"Other" includes all blobs with other Ground Truth classes ex- 
cept "Problem Field", "No Ground Truth", and "Mixture". 
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TABLE 4. LABELING RESULTS FOR 
ALL PIXELS COMBINED 



All Profiles Used 

No Grass or Flax Profiles 

Thresholds 

Thresholds 


Weighting 

0.0 

0.5 

0.667 

0.0 

0.5 

0.667 

1-0-0 

.68 

.71 

.71 

.62 

.71 

.71 

0-1-0 

.63 

.65 

.66 

.56 

.61 

.64 

0-0-1 

.63 

.64 

.64 

.48 

.57 

.57 

1-1-0 

.68 

.71 

.72 

.62 

.71 

.73 

1-0-1 

.70 

.73 

.73 

.53 

.71 

.73 

0-1-1 

.65 

.66 

.66 

.48 

.60 

.63 

1-1-1 

.69 

.72 

.73 

.54 

.72 

.74 

1-1-2 

.69 

.71 

.72 

.50 

.69 

.71 

1-2-3 

.68 

.70 

.70 

.49 

.65 

.68 
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On a segment-by-segment basis, however, accuracies varied con- 
siderably. Tables 5 and 6 and Figure 4 present these results. Neither 
year, agrophysical stratum unit, or the number of available acquisitions 
were shown to be significant with respect to grain labeling accuracy. 

This result is surprising, since logical connections exist between 
these factors and the labeling technique (see Section 3.2.1). One ex- 
planation for the lack of significance may be that the acquisition place- 
ment is critically important (see Section 4.4.2), and overshadowed the 
influence of the factors tested. 

4.3 LABELING ERROR CHARACTERIZATION 

4.3.1 ERRORS OF OMISSION 

Accuracy by Type of Grain Crop 

Only three of the spring small grains had enough data to support 
analysis: Spring Wheat, Barley, and Oats. Durum Wheat was only repre- 

sented by a few blobs in two segments, and therefore could not be 
evaluated. Of the three crops. Oats showed the worst labeling results, 
usually several percentage points below Wheat. When the test-statistic 
included all three features (Greenness, Brightness, and shift) Barley 
was most often correctly labeled, though again by only a few percentage 
points. Those Spring Wheat blobs that were correctly labeled, however, 
seemed to fit the grain profile better than did the other grain crops. 
When the probability threshold was increased from 0.0 to 0.5, the per- 
centage of Spring Wheat blobs still called grain decreased less than did 
the percentage for Barley or Oats. There were too few strip fallow 
grain blobs to allow separate evaluation of this category. 
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TABLE 5. SELECTED LABELING 
RESULTS BY SEGMENT 


All Profiles 
Weighting: 1-1-1 

(Only segments with 10 or more 
Grain Blobs Labeled) 


Threshold 

0.0 0.5 0.667 


Segment 

Grain 

Other 

Grain 

Other 

Grain 

Other 

1380 

.56 

.99 

.47 

.99 

.41 

.99 

1394 

.56 

.84 

.55 

.91 

.55 

.91 

1498 

.44 

.83 

.33 

.89 

.17 

.93 

1521 

.86 

.63 

.81 

.67 

.78 

.67 

1542 

.28 

.89 

.20 

.93 

.20 

.93 

1567 

.11 

.97 

.07 

.99 

.07 

.99 

1614 

.28 

.75 

.24 

.89 

.23 

.89 

1618 

.42 

.91 

.41 

.92 

.41 

.92 

1619 

.23 

.94 

.22 

.98 

.21 

.98 

1637 

.75 

.75 

.71 

.85 

.69 

.88 

1661 

.52 

.59 

.49 

.78 

.46 

.82 

1663 

•71 

.95 

.60 

.98 

.57 

.98 

1675 

.51 

.52 

.49 

.56 

.46 

.58 

1676 

.50 

.83 

.42 

.85 

.33 

.85 

1725 

.57 

.60 

.28 

.70 

.20 

.72 

1731 

.13 

.91 

.13 

.91 

.13 

.91 

1755 

.12 

.89 

.12 

.90 

.12 

.90 

1800 

.52 

.51 

.47 

.57 

.42 

.62 

1811 

.52 

.54 

.37 

.63 

.36 

.68 

1842 

.08 

.99 

.08 

1.00 

.08 

1.00 

1913 

.44 

.55 

.31 

.69 

.27 

.74 

1942 

.27 

.94 

.27 

.95 

.27 

.95 

1948 

.50 

.87 

.19 

.97 

.13 

.98 
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TABLE 6. SELECTED LABELING 
RESULTS BY SEGMENT 


No Grass or Flax Profiles 
Weight ing : 1-1-1 

(Only segments with 10 or more 
Grain Blobs Labeled) 


Threshold 




0.0 

0.5 


0. 

667 

Segment 

Grain 

Other 

Grain 

Other 

Grain 

Other 

1380 

.75 

.97 

.59 

.99 

.53 

.99 

1394 

.88 

.39 

.82 

.79 

.78 

.84 

1498 

1.00 

.08 

.61 

.79 

.39 

.89 

1521 

.97 

.25 

.91 

.42 

.88 

.50 

1542 

.79 

.42 

.64 

CM 

00 

• 

.44 

.85 

1567 

.21 

.92 

.14 

.97 

.11 

.98 

1614 

.99 

.03 

.76 

.72 

.65 

.75 

1618 

.92 

.55 

.87 

.79 

.86 

.81 

1619 

.57 

.66 

.53 

.92 

.51 

.96 

1637 

.98 

.11 

.87 

.75 

.80 

.84 

1661 

.96 

.07 

.57 

.73 

.49 

.78 

1663 

.95 

.87 

.78 

.98 

.71 

.98 

1675 

.92 

.06 

.82 

.22 

.72 

.34 

1676 

.50 

.83 

.42 

.85 

.33 

.85 

1725 

.96 

.05 

.57 

.22 

.44 

.29 

1731 

1.00 

.39 

.93 

.59 

.80 

.74 

1755 

.82 

.25 

.60 

.73 

.56 

.77 

1800 

.86 

.16 

.58 

.48 

.51 

.56 

1811 

.71 

.34 

.51 

.56 

.47 

.62 

1842 

.30 

.99 

.27 

1.00 

.27 

1.00 

1913 

.78 

.24 

.39 

.61 

.32 

.69 

1942 

.98 

.59 

.85 

.77 

.79 

.80 

1948 

00 

00 

• 

.45 

.22 

.96 

.13 

.98 
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FIGURE 4. ACCURACIES BY SEGMENT 


2erim 

Major Error Sources 

By far the most common confusion profile set for grains was that 
representing grass. In every configuration grass was the second most 
frequently chosen profile after grain. With no probability threshold, as 
much as 44% of the grain sample was assigned to the grass class (depend- 
ing on test-statistic weighting) . Flax was the second most commonly 
chosen confusion profile, attracting as much as 25% of the grain sample. 
Both of these results were expected, since the profile shapes for the 
three crops were very similar (see Figure 1). 

When the grass profile was omitted, grain labeling accuracies in- 
creased by about 15 percentage points, while omission of the flax profile 
resulted in an increase of 10 percentage points. When both profiles were 
omitted, grain labeling accuracy increased approximately 30 percentage 
points. While these figures varied somewhat with thresholds and test- 
statistic weightings, the trends were very stable. 

The other crop profiles were much less likely to draw away grain 
samples. Even when the grass and flax profiles were omitted from the 
procedure the corn, sunflower, and soybean profiles each drew only 2-6% 
of the grain sample. 

4.3.2 ERRORS OF COMMISSION 

The primary sources of error related to non-grains called grain 
were the same crops that captured most of the grain samples; grasses 
and flax. 

The grasses class, which was comprised of all blobs assigned ground 
truth labels of grass, hay, pasture, and idle fallow, had around 25% of 
its members labeled grain, or 50-75% if the grass profile was omitted. 
Broken down further, the four major elements of the class had approxi- 
mately the same accuracies. It should be noted, however, that the 
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pasture class comprised 65% of the total grass blobs, and 80% of the 
total interior pixels of this class. Thus in terms of absolute numbers, 
pasture was the most Important source of erroneous grain labels. A 
technique to detect pasture prior to the labeling process, perhaps based 
on the irregular field shapes typical of the class, could increase 
accuracy considerably. Table 7 shows the results obtained when Pasture 
blobs are designated Other. The results can be compared to those pre- 
sented in Tables 3 and 4. 

Flax was called grain 40-50% of the time, while 15-20% of the corn 
and soybeans blobs were labeled grain. Although 30-60% of the sunflower 
blobs were called grain, this may not accurately indicate the spectral 
confusion between these crops. Even sunflower data were not called sun- 
flowers very' frequently , suggesting that the sunflower profile set was 
not in fact a good representation of sunflower spectral development; the 
confusion, given a good sunflower profile, would probably have been less. 

Labeling accuracy for two other important crops was also evaluated: 
winter wheat and alfalfa. Although it is planted in the fall, the 
development of winter wheat in the spring is very similar to that of 
spring wheat, so one might expect that winter wheat would frequently be 
called a spring grain in this procedure. However, on the average, only 
about 20% of the winter wheat blobs were called spring grains. The 
majority of winter wheat blobs best fit the corn profile, and thus were 
labeled other. 

The cutting cycles of alfalfa can produce a wide range of spectral 
patterns for alfalfa blobs, some of which could easily be mistaken for 
grain. Nonetheless, two to three times as many alfalfa blobs were 
labeled grass as were labeled grain. Only about 20% of the alfalfa 
blobs received grain labels. However, this number increased substan- 
tially, to about 60%, when grass and flax profiles were omitted. 



TABLE 7. LABELING ACCURACIES WITH 
PASTURE DESIGNATED OTHER 
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4.4 QUALITATIVE COMPONENT EVALUATION 

4.4.1 CROP CALENDAR SHIFT 

The concept of crop calendar shift estimation relies on the assump- 
tion that the span of the growing season can be determined from the few 
scattered acquisitions available. In particular, it depends on the 
ability to detect a peak Greenness. Examination of histograms of the 
shift estimates by ground-truth class suggests that for the most part 
the technique perfomed as Intended, Figure 5 provides illustrations 
of "good" results, and the normal-like distributions one would expect 
for planting dates in a given region. 

In other segments, however, shift estimates for a given crop type 
varied considerably (Figure 6) . Such results tended to occur when there 
were few acquisitions available in the growing season, when the acqui- 
sitions were spaced such that large gaps in coverage were apparent, or 
when the crop of interest showed unusually little green development. 

All of these conditions affect the spectral features used to estimate 
crop calendar shift. While it could be the case that planting occurred 
in an unusual pattern in the particular segment, it is most likely that 
the algorithm was fooled by noise, too little information, or information 
of low quality. 

This conclusion can be confirmed by looking at the fit to the 
Brightness profile. While the shift estimate maximizes Greenness corre- 
lation, and thus should provide a somewhat reasonable Greenness fit, the 
fit to the Brightness profile, with a shape that differs substantially 
from that of the Greenness profile (at least for spring grains), would 
be expected to be worse with inaccurate shift estimates. Figure 7 illus- 
trates that for the spring small grains, that is indeed what happens. 

One sees a reduced incidence of high probability fits to Brightness as 
the deviation from the mean or expected shift increases. 
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FIGURE 5. CROP CALENDAR SHIFT HISTOGRAMS - GOOD RESULTS 
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FIGURE 6. CROP CALENDAR SHIFT HISTOGRAMS - POOR RESULTS 
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FIGURE 7. BRIGHTNESS CORRELATION AS RELATED TO DEVIATION FROM 
EXPECTED SHIFT. (Grain Data Fit to Grain Profile) 
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While the crop calendar shift procedure seems for the most part 
to be providing adequate information, it is limited by the spectral data 
available to it, and as a result is itself limited to segments which 
provide a good set of acquisitions. The problem of little or no green 
development will be addressed in the next section. 

4.4.2 PROFILES AND PROFILE FITTING 

Greenness 

The most basic assumption of the labeling technique presented in 
this report is that data of a particular crop type do follow a 
characteristic pattern of spectral development. In terms of Greenness, 
this assumption is clearly supportable. Figures 8 and 9 show shifted 
data for two crops from all the test segments combined. While indi- 
vidual trajectories are not traced in the figure, it is clear that the 
data as a whole do follow a general pattern, and indeed follow a differ- 
ent pattern based on crop type. Such results have been observed before 
[6,11]. Nevertheless not all blobs said to be of a given crop type in 
the ground truth information follow the expected pattern of Greenness 
development. Figures 10 and 11 illustrate this fact. Whether the 
cause of these spectral patterns is misregistration, ground truth errors, 
cultural factors such as abandonment and early cutting, or environmental 
factors such as drought and hail damage, the patterns deviate enough 
from a "normal" profile that the fields probably would not be detectable 
as grains in any procedure based on multitemporal spectral analysis. 

An additional issue raised by these data relates to their treatment 
in tests such as this, or in operational systems. Some odd profiles are 
the result of ground truth errors, and clearly "errors" in labeling 
such data may not be errors at all. Where misregistration is the cause 
of deviant spectral development patterns, the "true" label of a blob 
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is ambiguously defined. The incidence of such ambiguous labels can be 
minimized through use of a spectral-spatial clustering algorithm like 
SUPERB, but where they do occur, their treatment is a problem more com- 
plex than simply assigning the "correct" label. 

The third case is that in which cultural or environmental influences 
have substantially altered the physical and, therefore, the spectral 
character of the field. In many of these cases it is probably not de- 
sirable to label a field as a normal crop, since it will add little or 
nothing to the final production of the region. While this is dependent 
on the yield estimation component of the system, and is thus, to some 
degree, beyond the scope of this work, it again influences the evaluation 
of test results for labeling techniques. If grain fields that have been 
green chopped for silage or extensively damaged by hail are not going 
to contribute to the production figure, then the "correct" label for 
such fields may not be grain but rather non-grain. 

Brightness 

As previously described. Brightness development over time is a 
more complex process than Greenness development. The interaction of 
soil and plant canopy results in greater variability in spectral 
patterns exhibited by a single crop, and particularly in Brightness 
values early and late in the season. This situation is clearly illus- 
trated in Figures 12 and 13. At the same time, however, patterns in 
Brightness development are important for crop discrimination, as des- 
cribed in Section 4.5.2. 

There is clearly a need for more work in this aspect of the labeling 
process, in terms of characterizing the shapes of crop Brightness pro- 
files, detecting those shapes in data of unknown type, and adjusting ex- 
pectations in response to soil Brightness and crop condition changes. 
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Acquisition Dependence 

A key element of profile fitting already mentioned is the spacing 
and frequency of acquisitions. Profile-based procedures such as this 
one may work well or poorly, entirely as a function of the set of 
available data acquisitions. Two reasonably different profiles may 
look very different (Figure 14a) or identical (Figure 14b) depending 
on where the acquisitions fall. This problem, while very significant, 
is largely outside the control of any labeling procedure. It does, 
however, point out both the need for frequent coverage and an inherent 
limitation in multitemporal spectral analysis techniques. The problem 
may be aggravated by automated profile-based labelers, which must 
operate on a simpler level than the human mind, but it is a factor in 
all labeling techniques. 

4.5 EVALUATION OF PROCEDURE CONFIGURATIONS 

4.5.1 PROFILE SETS 

Section 3.3.2 describes the various sets of profiles evaluated. 
Previous sections have already discussed the influences of some of the 
described variations. The general trend through all eight sets tested 
was a decrease in errors of omission and an Increase in errors of com- 
mission as the number of non-grain profiles was reduced. This is par- 
ticularly true for the grass profile, which attracted many blobs from 
many other crop groups, as well as many grains. The best results, 
based either on combined grain/other accuracies or on the best mix of 
accuracies for the two crops, were achieved when both the grass and 
flax profiles were omitted. 

The subsetting intended to simulate that which could be done using 
historical statistics had the same effect as omission of other profiles, 
but the relative gain, weighting grain accuracy against non-grain accu- 
racy, was of uncertain significance. 
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4.5.2 TEST STATISTIC WEIGHTINGS 


Section 3.3.3 describes the various weightings evaluated. When 
used by themselves or in most pairs, the three features (Greenness, 
Brightness, and shift) yielded worse results than when all three were 
used. When only one was used, the best grain labeling accuracies were 
obtained with Brightness correlation, and the worst with Greenness fit. 

The explanation for the superiority of Brightness as a single discriminant 
may have to do with the Brightness dip which occurs in grains around 
the time of heading (see Figure 15) . This dip is not apparent in any 
of the other crops for which profiles were constructed. The dip proba- 
bly results from a combination of shadowing effects from the heads, 
and the more opaque quality of the heads. Although there are problems 
associated with Brightness as a discriminant (see Section 4.4.2), it 
nonetheless produced the best single feature results. 

Figure 16 illustrates a likely explanation for the low accuracies 
achieved with Greenness fit used alone. When shifted so that their 
peaks line up, the Greenness profiles of all the crops look very similar, 
varying primarily in their temporal spread. Since the tails of the pro- 
files tend to be the most noisy, it could be expected that many grain 
blobs would fit the longer profiles fairly well simply as the result of 
a higher than expected early or late Greenness value. 

In this test, grain labeling accuracies were maximized by those 
weightings which downplayed Greenness fit and emphasized Brightness 
correlation and shift: the 0-1-1 and 1-2-3 weightings achieved the 

best results (refer to Tables 3 and 4). These weightings were also 
best for maximizing overall accuracy (grain and non-grain) when all 
profiles were used. However, since the flax and grass profiles resemble 
grain most in Greenness, omission of one or both of these profiles 
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FIGURE 15. SMALL GRAINS BRIGHTNESS PROFILE 
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FIGURE 16. COMPARISON OF GREENNESS PROFILES 
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allowed the emphasis on Greenness fit to be increased. When both were 
excluded, the 1-1-1 and 1-1-2 weightings produced the best results 
(see Tables 3 and 4). 

4.5.3 PROBABILITY THRESHOLDS 

Any increase in the probability threshold will almost certainly 
reduce the number of blobs assigned to a crop code. In terms of grain 
vs. non-grain, a higher probability threshold will result in a purer 
sample (decreased errors of commission) but also increase errors of 
omission. 

Of the thresholds considered (see Section 3.3.4), the 0.5 and 0.667 
values were found to be best in terms of both accuracy with which a 
crop was called Itself (grain called grain, corn called corn, etc.) 
and overall grain/non-grain accuracy. 

The incremental change in grain accuracy can be compared to that 
in non-grain accuracy as a further evaluation of the probability thres- 
holds. Comparison of the 0.0, 0.5, and 0.667 thresholds showed a sub- 
stantial gain achieved when going from the 0.0 to the higher levels, 
particularly in configurations excluding the grass and flax profiles. 

The incremental gains or losses incurred by moving from the 0.5 to 0.667 
levels appear insignificant, although the highest total labeling accu- 
racies for all data regardless of type were achieved with the 0.667 
threshold. In terms of overall grain and non-grain accuracies, the best 
threshold was either 0.5 or 0.667, depending on the test statistic 
weighting used (see Tables 3 and 4). Different effects were also ob- 
served between individual segments (Tables 5 and 6). 
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CONCLUSIONS 


A labeling technique requiring minimal analyst resources has been 
developed and tested on an extensive and independent data set. Results 
of that test suggest that the technique, as it is currently defined, can 
achieve spring grain labeling accuracies similar to those achieved with 
analyst-intensive techniques, though with lower non-grain accuracies. 

Among the procedure configurations tested, the best labeling 
accuracies were obtained using: 

Profile Sets: Exclude grass and flax 

Test-Statistic Weighting: 1-1-1 

Probability Threshold: .667 

Some other statistic weightings, particularly those that increase 
emphasis on Brightness, should be considered in the future. 

Test results also suggest that the accuracy of the technique could 
be substantially improved through particular modifications. These 
modifications, and some implications of the test for profile-based 
labelers and multitemporal labeling techniques in general, are described 
below. 

Modifications Suggested 

As detailed in Section 4.3.2, the pasture class is a major source 
of erroneous grain labels. As suggested in that section, a fairly 
simple technique, based on either visual or digital detection of field 
shapes, could be used to identify the pasture class prior to labeling, 
and so allow substantial improvements in non-grain labeling accuracy. 

It is clearly desirable that such a technique be developed and added 
to the labeling procedure as described in this report . 



The second major area of interest Is that of characterizing and 
using Brightness development patterns in labeling. There are several 
obstacles to effective use of this information, but the importance 
of Brightness as a discriminant feature, described in Section 4.4.2, 
provides ample incentive to address the problems. 

As a whole, our understanding of Brightness and its relationship 
to crop physical characteristics is less developed than that of Greenness. 
This understanding, and the resultant development of techniques for de- 
tecting soil Brightness and crop condition and using that Information 
to adjust expected crop Brightness profiles, could allow substantial 
Increases in the ability of this or similar labeling techniques to 
accurately detect grains, and perhaps other crops as well. 

Other Implications 

The test and evaluation of this labeling procedure on a large 
data set has raised several issues which relate to the whole discipline 
of crop identification using Landsat. First, only about half of the 
blobs in the entire data set of 64 segments met the acquisition require- 
ments for labeling. One might conclude from this fact that profile-based 
techniques are impractical for use in area estimation systems due to 
their acquisition constraints. We suggest, however, that the mathe- 
matical requirements imposed by this procedure are in fact indications 
of practical requirements for any labeling procedure, that while an 
analyst or simple classifier may produce labels using less information 
than the three acquisitions required by this technique, they will be 
unable to produce accurate labels in most such cases. Others have ob- 
served similar limits [9,10]. 

Clearly, then, there is a need to design systems which can provide 
more frequent coverage, and also to develop procedures that can extract 
the maximum information possible from the limited set of available 
acquisitions . 



A second implication relates to any multitemporal analysis tech- 
nique which assumes characteristic patterns of temporal-spectral develop- 
ment for crop classes. Results described earlier clearly indicate that 
at least some data of a particular crop class (as identified in ground 
data) show little if any similarity to the expected spectral develop- 
ment pattern. Those deviations caused by errors in the ground data or 
misregistration of data between dates are beyond the control of labeling 
techniques, but are probably an insubstantial portion of the total. The 
remainder of the unexpected patterns either point out an inherent flaw 
in the pattern matching approach, or suggest the need for finer defini- 
tion of the crop classes. While there is a range of spectral development 
patterns one would expect from normal fields of a given crop type, our 
understanding of the effects of crop variations on spectral development, 
particularly with regard to Tasseled-Cap Greenness, leads us to conclude 
that the deviant patterns observed represent not a normal range of varia- 
bility, but the result of drastic cultural or environmental events (e.g. , 
green chop for silage, abandonment, hail damage). Fields altered so 
significantly will contribute little if anything to regional crop pro- 
duction, and as a result, are probably best assigned to the non-grain 
(or non-crop of interest) category. Thus it can be said that this and 
similar techniques best detect "producing grains" (or other crops) 
rather than all fields containing plants of a given species. As an 
input to a crop production estimation system, this is probably more 
useful information. 

The test and evaluation reported herein has provided an indication, 
though not an absolute proof, of the utility of multitemporal profiles 
and related features as the basis for automatic labeling techniques. 
Techniques such as this one, which use the integrative powers of the 
human analyst to provide a context within which a more efficient and 
objective computerized technique can assign labels, show great promise 
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for advancing the accuracy and efficiency of this most difficult step 
in crop area estimation systems. With the understanding provided by 
this and similar tests, combined with continual advances in our under- 
standing of features in Landsat data and their relationship to crops and 
crop condition, substantial Improvements in the state-of-the-art of 
labeling should be possible. 
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NOTE: Where more than one Brightness profile are 
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